A Naïve Bayes classifier for Shakespeare's second-person pronoun

In order to investigate in explicit detail the way that yand thpronouns alternate in the Shakespearean corpus, I have undertaken a collocational analysis of the full corpus of Shakespeare’s 37 plays and found that (1) second-person pronouns can be disambiguated based on context alone, (2) ypronouns seem to be used in more formal situations or when an inferior is addressing a social better, and (3) the thpronoun is reserved for addressing peers, servants, or other familiar personages. Through the Python Natural Language Toolkit (Bird et al., 2009, Natural Language Processing with Python. Sebastopol, CA: O’Reilly Media), I implemented a Naı̈ve Bayes classifier that in effect treats each occurrence of a second-person pronoun as a black box that must be resolved into either a ypronoun or a thpronoun based only on the surrounding words. Using tenfold cross-validation, the classifier achieves an accuracy of 78.3% when fellow thand ypronouns are excluded from the context and 88.0% when we allow fellow thand ypronouns to assist in classification. Most interesting, however, are the context words that prove most informative in categorizing the pronouns. Significantly, the words most useful in classifying a pronoun as a ypronoun include high-register words such as lordship, madam, lords, and sir. After a group of conjugated second-person verbs like art and wert, the words most associated with thpronouns are words such as torment, nuncle, lesser, and villain. The ability to discriminate between forms based only on context confirms the hypothesis that the two classes of second-person pronoun are indeed used distinctly in the Shakespearean corpus. The list of words most helpful in making that distinction strongly suggests a difference in formality. We can also gain additional insight into the plays by examining some of the unexpected words that collocate with either one form or the other. .................................................................................................................................................................................